Multiple-choice questions are a common way of testing students in a variety of subject areas. In most cases, a multiple-choice question comprises three identifiable sections: a section containing a set of facts to be presumed (for instance, a narrative, a short story, a poem, an expression, an equation, or a geometric figure), an interrogative sentence (sometimes known as the “call of the question”), and a set of answer choices.
A multiple-choice question may conveniently be divided into two parts—a first part, comprising a set of facts to be presumed and an interrogative sentence, and a second part, comprising a set of answer choices. The first part may also be termed a “query.” (The term “query” can alternatively refer to the interrogative sentence alone, but as used herein, the term “query” refers to the entire first part of the question (i.e., both the set of facts to be presumed and the interrogative sentence), unless otherwise noted. In the second part, between three and five answer choices are typically presented, although the number of answer choices may vary below three or above five under appropriate circumstances. (For instance, in a so-called “true/false question,” there are typically two answers: “true” and “false.”)
The set of facts to be presumed may be expressed using words or phrases, or a set of objects or symbols, or a combination of words, objects and symbols. Alternatively, the set of facts to be presumed may be expressed in any other appropriate way, such as with a figure, a picture, or another graphical representation. (For instance, in an art history exam, the set of facts to be presumed may constitute a piece of art or a picture thereof.) The interrogative sentence typically asks the student or examinee to pick the “correct,” or the “best,” answer, and to indicate the selected answer choice either on the exam paper, for instance, by circling the selected answer choice, or on a separate answer sheet. The separate answer sheet may include a small shape, such as a small circle, oval or rectangle, corresponding to each answer choice of each question, which shape may be filled in by the examinee, for instance with a pencil. Typically, the examinee is asked to fill in the shape corresponding to the selected answer, while leaving blank the shapes corresponding to the question's other answer choices. Other answer sheets may have any other appropriate configuration now known or later developed. In most cases, the examinee may leave a question unanswered, but may not select more than one answer choice per question. Thus, a multiple-choice question generally has no more than one valid answer. In other cases, where multiple valid answers exist, various answers may yield either full credit or varying amounts of partial credit, and methods disclosed herein may be extended in a recursive, analogous manner.
Indication of answers on a separate answer sheet is popular, being suitable for automatic or machine grading of the answer sheets; the automatic or machine grader compares a given answer sheet with the template of “correct” answers and counts the number of questions where the filled-in shapes (for instance) match the template. The examinee's score, also sometimes called the “grade,” may then be computed based on a formula that may depend on the number of questions answered correctly versus the number of questions answered incorrectly. Undoubtedly, the low cost, high speed, convenience and uniformity with which multiple-choice tests can be graded contribute to their popularity.
Implicit in the typical scoring formulas for multiple-choice tests is an assumption that examinees with a mastery of the subject matter will work efficiently to select the correct answers, whereas those who depend largely on guesswork will not do much better than the statistical odds of hitting the correct answers at random. However, in practice, the distinctions between the examinees' scores are rarely as clear-cut, due to quirks of the multiple-choice format. Specifically, an examinee unfamiliar with the tested material can beat the statistical obstacles to a high score by relying, at least in part, on guesswork—if the examinee is able to eliminate one or more incorrect answer choices and “guess” from a smaller pool of possible answers. On the other hand, an examinee who understands the tested material well may inadvertently choose an incorrect answer, despite that understanding, because of a minor error in analysis or computation. Such an error, no matter how minor, will lead to a complete loss of credit for the question, if it results in selection of an incorrect answer choice. In short, the typical multiple-choice testing format provides little room for demonstrating the soundness of the underlying analysis or the accuracy of computation except in the final answer.
In a common scenario, total loss of credit may result from, for example, an incorrect answer choice being selected in lieu of the correct answer choice, based on the distinction of a single word, phrase, value or fact in the question (even though the examinee understood the question). For example, if a multiple-choice question asks an examinee to calculate the volume of a box having sides of 1 foot, 2 feet and 3 feet, the correct answer is the product of the three lengths, or 6 cubic feet. An examinee who understands the question may inadvertently choose the incorrect answer “9 cubic feet” if he or she misreads the “2” as a “3.” Thus, the very ease and simplicity of selecting and recording an answer to a multiple-choice question may obscure the difference between knowledge and ignorance on the examination.
This quirk of the multiple-choice format leads to a gap between classroom teaching, at nearly all levels of instruction and in all subject disciplines, on the one hand, and multiple-choice examinations, on the other hand. Teaching in a manner calculated to prepare students for a multiple-choice exam (“teaching to the test”) requires devoting class time to test-taking “tricks,” reducing the time available for actual teaching of the subject matter. In addition, focusing on “tricks” diverts attention from the aforementioned problem with multiple-choice exams, which cannot be solved with such conventional tricks, but requires instead a teaching method that instills in students the discipline to carefully analyze the most important facts or decision-points within a question.
Despite the aforementioned difficulties, the multiple-choice testing format is widely used, particularly for national and standardized tests. The popularity of multiple-choice testing has given rise to a need for educational products that provide students with substantive test-taking preparation. At present, substantial publishing resources are devoted exclusively to preparing students to take various multiple-choice examinations. These resources typically rely on various permutations and combinations of the following paradigms: state important subject matter concepts; emphasize and reinforce key concepts; present a number of multiple-choice questions with answers, either with or without explanations of why the answer choices are right or wrong; append simulated examinations; and, present a collection of general strategies, tips and tricks for answering multiple-choice questions.
Such materials primarily seek to familiarize students with the multiple-choice format in general and the types of questions in the subject or discipline which can be asked in such a format, and in varying degrees, to reinforce the subject matter concepts.
In terms of practical guidance for answering each question, the currently available materials offer three models: provide the correct answer without explanation; provide the correct answer with an explanation of why it is correct; or, provide the correct answer with explanations for each answer choice to show why it is correct or incorrect.
However, the currently available materials, even those providing explanations for each answer choice, leave a gap in teaching the art of closely reading the fact pattern of a question, critically evaluating the answer choices, and learning to recognize the critical pieces of information in the fact pattern on which these answer choices turn. For this reason, these materials are not designed to exploit the unique strength of the multiple-choice format as a powerful device to learn and absorb any subject matter.